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1 Introduction 



J. L. Doob was a pioneer in the development of the theory of martingales and its applications to 
probability theory, potential theory or functional analysis. The fundamental contributions that 
he made in this field form the cornerstones of one of the richest veins explored in mathematics 
during the last half-century. In particular, martingales and stochastic calculus provide nowadays 
key tools for studying the asymptotic behavior of random processes; see the classical books by 
Ethier and Kurtz [10] and Jacod and Shiryaev [12]. In the present work, we shall apply such 
techniques to investigate a class of stochastic flows related to certain population dynamics. 

The general motivation for the present work is to get a better understanding of the relations 
between the so-called coalescents with multiple collisions, which were introduced independently 
by Pitman [16] and Sagitov [17], and continuous-state branching processes. Note from [17] that 
coalescents with multiple collisions can be viewed as asymptotic models for the genealogy 
of a discrete population with a fixed size, and so the existence of connections with branching 
processes should not come as a surprise. Such connections were already derived in [3] , where the 
Bolthausen-Sznitman coalcsccnt was shown to describe the genealogical structure of a particular 
continuous-state branching process introduced by Neveu, and in [6] , where similar relations were 
obtained between the so-called beta-coalescents and continuous-state branching processes with 
stable branching mechanism. Here, we do not focus on exact distributional identities, but rather 
on asymptotics for functionals of coalescent processes, where the limiting objects are given in 
terms of branching processes. In order to get such asymptotics, we apply the machinery of limit 
theorems for semimartingales [12] to the so-called generalized Fleming- Viot processes, which 
where shown in [4] to be duals to the coalescents with multiple coUisions. 

Generalized Fleming- Viot processes, which model the evolution of a continuous population 
with fixed size 1, have appeared in articles by Donnelly and Kurtz [7, 8], and were studied 
more recently in our work [4, 5]. It is convenient to view a generalized Fleming- Viot process 
as a stochastic flow {Ft^t > 0) on [0,1], such that for each t > 0, Ft : [0,1] [0,1] is a 
(random) right-continuous increasing map with Ft{0) = and Ft{l) = 1. We should think of 
the unit interval as a population, and then of F^ as the distribution function of a (random) 
probability measure dFt(x) on [0, 1]. The evolution of the latter is related to the dynamics of 
the population as follows : For every < ri < r2 < 1, the interval ]Ft{ri), Ft{r2)] represents 
the sub-population at time t which consists of descendants of the sub-population ]ri,r2] at the 
initial time. The transitions of the flow arc Markovian, and more precisely, for every s,t > 0, we 
have Ft+s = Fg o Ff, where Fg is a copy of Fg independent of (F^, < r < t). The distribution 
of the flow is then characterized by a measure u on ]0, 1] such that Jp^x'^u{dx) < oo. To 
explain this, consider the simple case where v is a flnite measure. Let ((Tj, Ui, G N) denote 
the sequence of atoms of a Poisson random measure on [0, oo[x[0, 1] x [0,1] with intensity 
dt®du® u^dx), ranked in the increasing order of the flrst coordinate. The process {Ft, t > 0) 
starts from Fq — Id, remains constant on the intervals [Tj_i, (with the usual convention that 
To = 0), and for every i e N 

Ft, = A, o Ft,_, 

where 

Ai{r) = ^il{u,<r} + r{l - 6) , re [0, 1]. 
In terms of the population model, this means that at each time Tj, an individual in the popu- 
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lation at time Ti_i is picked uniformly at random and gives birth to a sub-population of size 
^j. Simultaneously, the rest of the population shrinks by factor 1 — ^j, so the total size of 
the population remains 1. The previous description does not apply when v is infinite, since 
then the Poisson measure will have infinitely many atoms on a finite time interval. Still, the 
Fleming- Viot fiow can be constructed via a suitable limiting procedure ([4] Theorem 2). 

Our first motivation for studying generalized Fleming- Viot processes came from their re- 
markable connection [4] with the class of coalescents with multiple collisions considered by 
Pitman [16] and Sagitov [17]. To describe this connection, fix some time T > viewed as the 
present date at which the population is observed, and pick a sequence of individuals labelled 
1,2,... independently and uniformly over [0, 1]. For every t < T, we obtain a partition n(t) of N 
by gathering individuals having the same ancestor at time T — t. The process (n(t), < t < T) 
is then a Markovian coalescent process on the space of partitions of N. In the terminology of 
[16], it is a A-coalescent, with kidx) — x^v{dx\ started from the partition of N into singletons. 
As a consequence of Kingman's theory of exchangeable partitions, for every t > 0, each block of 
n(t) has an asymptotic frequency, also called the size of the block, and the ranked sequence of 
these frequencies yields a Markov process called the mass-coalescent. As a consequence of the 
preceding construction, the mass-coalescent at time t has the same distribution as the ranked 
sequence of jump sizes of Ft- 

The first purpose of the present work is to investigate the asymptotic behavior of a rescaled 
version of the preceding population model. Specifically, we consider a family (z>("\ a > 0) of 
measures on ]0, 1] such that /]q x'^u^'^\dx) < oo for every a > 0, and the associated generalized 

Fleming- Viot processes F^"). For each a > 0, we rescale F*^"^ by a factor a in space and time, 
i.e. we set 

F/")(r) := aF^\r/a), r G [0,a],t>0. 

So the process F^"'^ describes the evolution of a population with fixed size a. Roughly speaking, 
considering F^") in place of F^"'^ enables us to focus on the dynamics of a sub-population having 
size of order 1/a. Denote by u^'^^ the image of i>^"'^ under the dilation x ax, and assume 
that the measures {x^ A x)i'^"'\dx) converge weakly as a — > oo to a finite measure on ]0, oo[, 
which we may write in the form (x^ A x)n(dx). Then Theorem 1 shows that F^""^ converges in 
distribution to the critical continuous-state branching process Z with branching mechanism 

^{q) = [ (e-«^ - 1 + qx)n{dx) , q>0. 

J]0,oo[ 

As a consequence of this limit theorem, we derive a hydrodynamic limit for the associated 
coalescent processes (Theorem 2). Precisely, we show that under the same assumptions as 
above, for every t > 0, the empirical measure corresponding to the jumps of F^"''' (or equivalently 
to the block sizes in the associated coalescent) converges, modulo a suitable rescaling, towards 
a deterministic measure Aj. Informally, is the distribution ofa cluster at time t, that is 
a collection of individuals sharing the same ancestor at the initial time, in the continuous- 
state branching process with branching mechanism In a way analogous to the derivation 
of Smoluchovski's coagulation equation from stochastic models (see Aldous [1], Norris [15] and 
the references therein for background) we prove that the family {Xt,t > 0) solves a generalized 
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coagulation equation of the form 

dt A;! J]o,oo['= 

where / can be any continuous function with compact support on ]0, oo[ (Proposition 3). 

In the last part of this work, we study the small time behavior of generalized Fleming- Viot 
processes and A-coalescents, under a regular variation assumption on the measure u (recall that 
A{dx) = x'^uiydx)). Precisely, we assume that the tail ^{[s, 1]) is regularly varying with index 
—7 when e goes to 0. We are interested in the case when the A-coalescent comes down from 
infinity (i.e. for every f > 0, has finitely many blocks), which forces 1 < 7 < 2. Leaving 
aside the boundary cases we suppose that 1 < 7 < 2. As a consequence of Theorem 1, we prove 
that the rescaled Fleming- Viot process 

converges in distribution to the continuous-state branching process with stable branching mech- 
anism: 

We then use this result to investigate the small time behavior of the size of blocks in the A- 
coalescent. Write A^t(]0,a;[) for the number of blocks with size less than x in the A-coalescent 
at time t. If g{e) = {eulle, l]))^"*^, Theorem 4 states that 



sup 

a;G]0,oo[ 



£Ar,(,)(]0,£x[)-Ai(]0,x[) ^0, 



in probability. Furthermore, the measure Ai can be characterized by its Laplace transform 

(1 - e-^'-)Ai(dr) = (r(2 - 7) + gi-^)V(i-7). 



Theorem 4 is analogous to a classical result for the sizes of blocks in the Kingman coalescent 
in small time (see Aldous [1]). The proof uses an intermediate estimate for the total number 
of blocks in a A-coalescent, which is closely related to the recent paper [2] deahng with beta- 
coalescents. 

The paper is organized as follows. Section 2 gives a few preliminary results about continuous- 
state branching processes. In particular, the Poisson representation (Proposition 2) may have 
other applications. Section 3 states our first limit theorem for generalized Fleming- Viot pro- 
cesses. The derivation of the hydrodynamic hmit is developed in Section 4, which also discusses 
the generalized coagulation equation for the family {\t,t > 0). Finally Section 5 is devoted to 
the behavior in small time of generalized Fleming- Viot processes and A-coalescents. 

Notation. We use the notation (/i, /) for the integral of the function / with respect to the 
measure fi. We denote by Aip the space of all finite measures on ]0, oo[, which is equipped with 
the usual weak topology. We also denote by A^r the space of all Radon measures on ]0, oo[. 
The set A^r is equipped with the vague topology: A sequence (/x„,n e N) in AIr converges 
to II & A/r if and only if for every continuous function / :]0,oo[— > R with compact support, 
lini„_oo(/^n,/) = (/^,/)- 
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2 Stochastic flows of branching processes 



In this section, we give a few properties of continuous-state branching processes that will 
be needed in the proof of our limit theorems. A critical branching mechanism is a function 
^ : [0, oo[^ [0, oo[ of the type 

^(g) = + / fe-''« -1 + rq) 7r(dr) (1) 

i]0,oo[ ^ ^ 

where (3 > is the so-called Gaussian coefficient and tt is a measure on ]0, oo[ such that 

J{r Ar'^)'K{dr) < oo. The continuous- state branching process with branching mechanism ^ (in 
short the ^-CSBP) is the Markov process with values in M_|_, whose transition kernels Qt{x, dy) 
are determined by the Laplace transform 

jQtix,dy)e-''y ^exp{-xutiq)) , x,t>0,q>0, (2) 

where the function Ut{q) solves 

dut{q) 



dt 



-^K(g)) , Uo{q) = q. (3) 



The criticality of ^ imphes that a ^^-CSBP is a nonnegative martingale. If and Z'^ are 
two independent \E'-CSBP's started respectively at xi and X2, then Z^ -\- Z"^ is also a ^-CSBP, 
obviously with initial value Xi + X2. From this additivity or branching property, we may 
construct a two-parameter process Z = {Z{t,x),t,x > 0), such that: 

• For each fixed x > 0, {Z{t,x),t > 0) is a ^^-CSBP with cadlag paths and initial value 

Z{0, x) — X. 

• If Xi,X2 > 0, Z{-,xi + X2) — Z{-,xi) is independent of the processes {Z{-,x),0 < x < Xi) 

and has the same law as X2). 

These properties entail that for each fixed t > 0, Z(t, ■) is an increasing process with inde- 
pendent and stationary increments. Its right-continuous version is a subordinator with Laplace 
exponent Ut determined by (2) and (3). By the Levy-Khintchin formula, there exists a unique 
drift coefficient dt > and a unique measure A* on ]0,oo[ with /]o,(x)[(l ^ x)Xt{dx) < 00 such 
that 

ut{q) ^ qdt+ (1 - e-^^)At(cix) , g > . (4) 

J]0,oo[ 

One refers to At as the Levy measure of Z(t,-). Measures A^ play an important role in this 
work. Informally, we may say that A^ is the 'distribution' of the size of the set of descendants 
at time t of a single individual at time 0. This assertion is informal since A^ is not a probability 
distribution (it may even be an infinite measure). A correct way of stating the above (in the 
case dt = 0) is as follows: Z{t,x) is the sum of the atoms of a Poisson measure with intensity 
xXt{-). Moreover, the study of the genealogical structure of the ^^-CSBP (see e.g. [9]) allows 
one to interpret each of these atoms as the size of a family of individuals at time t that have 
the same ancestor at the initial time. 



5 



Prom now on, we assume that (3 — and we exclude the trivial case n — 0. 

We start by recalling in our special case an important connection between continuous-state 
branching processes and Levy processes due to Lamperti [14]. Let x > he fixed, and let 
C = {Ct, t > 0) denote a real-valued Levy process with no negative jumps, started from = x, 
and whose Laplace exponent is specified by 

E [eM-QiCt - 6))] = expm{q) , g > . 

In particular tt is the Levy measure of ^. The criticality of the branching mechanism \& ensures 
that the Levy process ^ has centered increments and thus oscillates. In particular the first 
passage time C := inf > : = 0} is finite a.s. Next, introduce for every t > 

7(i) = / - , a = inf{s>0:7(s)>OAC- 

Jo 4s 

Then the time-changed process oCt,t> 0) has the same distribution as {Z{t, x),t > 0). 

It follows from this representation that {Z{t,x),t > 0) is a purely discontinuous martingale. 
We can also use the Lamperti transformation to calculate the compensator of the jump measure 
of this martingale. By the Levy-Ito decomposition, the compensator of the jump measure of ^, 

is dt <S> Tr{dx). By a time change argument, we can then deduce that the compensator of the 
measure 

{t:AZ(t,x)^0} 

is Z{t, x)dt ® n{dr). 

Since {Z{t, x), t > 0) is a purely discontinuous martingale, the knowledge of the compensator 
of its jump measure completely determines the characteristics of this semimartingale, in the 
sense of [12] Chapter II. We will need the fact that the distribution of {Z{t,x),t > 0), and 
more generally of the multidimensional process {{Z{t, xi), Z{t, X2), . . . , Z{t, Xp));t > 0) for any 
choice of p and xi, . . . ,Xp, is uniquely determined by its characteristics. 

Fix an integer p > 1 and define 

Dp := {x = {xi, . . . , Xp) '■ < xi < X2 < • • • < Xp}. (5) 

For every (yi, . . . ,yp) G Vp, define a cr-finite measure U{yi, ■ ■ ■ ,yp; dzi, . . . , dzp) on ]R?J_\{0} by 
setting, for any measurable function ip : M.^ — >• M+ that vanishes at 0, 

00 

duip{rl{u<y,},---,rl{u<y^}). (6) 

Proposition 1 Let {xi, . . . , Xp) e Vp and let {Z^, . . . , Z^) be a p-dimensional semimartingale 
taking values in Vp, such that {Zq, . . . , Z^) — {xi, . . . , Xp). The following two properties are 
equivalent: 



J U{yi, ...,yp; dzi, dzp) ip{zi, ...,Zp)^ J n{dr) J 
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(i) The processes {{Zj , . . . , Zf);t > 0) and {{Z{t,xi), . . . , Z{t,Xp));t > 0) have the same 

distribution. 

(ii) The process {{Zl, . . . , Zf);t >0) is a purely discontinuous local martingale, and the com- 
pensator of its jump measure is the measure 

9{dt, dzi . . . dzp) — dt U {Z^, . . . , Zf; dzi, . . . , dzp). 



Proof: The implication (i)^(ii) is a straightforward consequence of the remarks preceding the 
statement and the branching property of continuous- state branching processes. We concentrate 
on the proof of the converse imphcation (ii)=^(i). Let q = {qi, ■ ■ ■ ,qp) e]0, oop, and let Yt = 
(y/, . . . , Yf) be defined by Y; = Zi - Zt^ if i > 2 and = Z^. Notice that Y; > 0. Using 
property (ii), an application of Ito's formula (cf Theorem II. 2. 42 in [12]) yields that the process 



exp(-g • Yt) - exp(-g • Yq 







- E / exp(-5 • Ys) (e''^'^ - 1 + qA l{„<yi}ds du 7r(dr) 

is a local martingale. This local martingale is bounded over the time interval [0, t] for any t >0, 
hence is a martingale. Taking expectations leads to 



E[e-«-^*] = E[e-^-^°] + V^fgJ f' dsElY^' e"'-^^]. 



(7) 



It is immediate to verify from (ii) that each is also a nonnegative local martingale, and so 
E[F;] < ¥.[Y^] =Xi- Xi_i (by convention = 0). If we set ft{q) = E[e-''-'^*] we have 

^:^ = -E[y/e-^-^*] 



dq, 

and so we deduce from (7) that 

dm 

dt 



+ *(g)-V/t(g) = 0, (8) 



where we write \E'(g) = (\l'(gi), . . . , \l/(gp)). In order to solve (8), fix ti > 0, and consider the 
function g{t) = {ut^^tiQi), ■ ■ ■ yUt^-tilp)) for t G [0,ti], where Ut{q) is as in (3). Since 

it follows that 

dftog dft , 

—gf- = -g^°9 + g[t)- V/t(^(0) = 

by (8). Hence ft o g{t) is constant over [0,ti], and 



ftiiq) = fhigiti)) = /o(^(0)) = exp{-^{xi - Xi_i)ut,{qi)). 



p 

i=l 
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This shows that 

0^t\^---, ^1) - 2^1), Z{tx, X2) - Z{ti, xi),..., Z{ti, Xp) - Z{ti, Xp_i)) 

and so 

(4, . . . , ZfJ {Z{h,Xi), Z{t,,X2), Z{ti, Xp)). 

It is easy to iterate this argument to obtain that the processes {{Z^ , . . . , Zf);t > 0) and 
{{Z{t,xi), . . . , Z{t,Xp));t > 0) have the same finite-dimensional marginal distributions. The 
desired result follows since both processes have cadlag paths. □ 

We now turn our attention to the representation of critical CSBP as stochastic fiows on 
[0, oo[ solving simple stochastic differential equations. On a suitable filtered probability space 
(n,^, (jFf),P), we consider : 

• an (^t)-Poisson random measure 

00 

i=l 

on R_|_ X [0, oo[x]0, oo[, with intensity dt®du® n{dr). 

• a collection {Xt{x),t >0),xE IR+ of cadlag (.7-t)-martingales with values in M_|_, 

• the stochastic differential equation 

Xt{x) ^ x+ M{ds,du,dr)rliu<x,-{x)} ■ (9) 

J[0,t]x[0,oo[x]0,oo[ 

The Poissonian stochastic integral in the right-hand side should be understood with respect to 
the compensated Poisson measure M (see e.g. Section II. 1 of [12]). This stochastic integral is 
well defined according to Definition II. 1.37 of [12], since the increasing process 



M{ds, du, dr) l{u<Xs-ix)} 

t]x[0,oo[x[0,oo[ 



is locally integrable under our assumption on tt. 

A pair (M, (X(a), a > 0)) satisfying the above conditions will be called a weak solution of 
(9). 

Proposition 2 The equation (9) has a weak solution which satisfies the additional property that 
^t{xi) < Xt{x2) for every t >0, a.s. whenever < xi < X2. Moreover, for every such solution 
(M, X), for every p e N and < Xi < . . . < Xp, the process {{Xt{x-\), . . . , Xt{xp)),t > 0) has 
the same distribution as {{Z{t, Xi), . . . , Z{t, Xp)),t > 0). 



Proof: The second part of the statement is immediate from the implication (n)=^(i) in Propo- 
sition 1. The first part can be deduced from Theorem 14.80 in [11] by the same arguments that 
were used in the proof of Theorem 2 in [5]. We leave details to the reader as this result is not 
really needed below except for motivation. □ 
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3 Generalized Fleming- Viot flows and their limits 

We now recall some results from [4, 5] on generalized Fleming- Viot processes and related 
stochastic flows. Let v denote a cr-finitc measure on ]0, 1] such that /jg y x^vidx) < oo. Ac- 
cording to Section 5.1 in [4], one can associate with u a Feller process {Ff.t > 0) with values 
in the space of distribution functions of probability measures on ]0, 1] (i.e. for each t >0, Ff is 
a cadlag increasing map from [0, 1] to [0, 1] with Ft{0) — and Ft{l) = 1), whose evolution is 
characterized by v and has been described in Section 1 in the special case when v is finite. 

In [5], we have shown that such generahzed Fleming- Viot processes can be described as 
the solution to a certain system of Poissonian SDK's. More precisely, on a suitable filtered 
probability space (^t),P), one can construct the following processes: 

• an (^4)-Poisson point process on ]R+x]0, l[x]0, 1] with intensity dt® du® u^dr), 

• a collection (Yt{x),t > 0), x G [0, 1], of adapted cadlag processes with values in [0, 1] with 
^t(a^i) < ^t(a^2) for all t > a.s. when < a;i < 0:2 < 1, 

in such a way that for every r e [0, 1], a.s. 

Yt{x) = x + f N{ds,du,dr)r (l{,<y^_(,)| - n_(a;)) . (10) 

■/[o,t]x]o,i[x]o,i] V ^ - ^ y 

The Poissonian stochastic integral in the right-hand side should again be understood with 
respect to the compensated Poisson measure N. 

Weak uniqueness holds for this system of SDK's (Theorem 2 in [5]). Furthermore, for every 
integer p > 1 and every < Xi < . . . < Xp < 1, the processes {(Yt{xi), . . . , Yt{xp)),t > 0) and 
{{Ft{xi), . . . ,Ft{xp)),t > 0) have the same distribution. Note the similarity with Proposition 
2: Compare (9) and (10). This strongly suggests to look for asymptotic results relating the 
processes Z{t,x) and Ft{x). 

For every integer p > 1 and every a > 0, set 

V^^Vpr\[0,a]P = {{xi,. . .,Xp) eRl -.0 <xi <X2 < . . . <Xp < a}. 

From (10) we see that for every (xi, . . . ,Xp) e P^, the process {Ft{xi), . . . , Ft{xp)) is a purely 
discontinuous martingale, and the compensator of its jump measure is 

dt R{Ft{xi), Ft{xp); dzi, dzp), 

where for every (yi, . . . ,yp) e V^, the measure R{yi, . . . ,yp; dzi, . . . , dzp) on M^\{0} is deter- 
mined by 

J R{yi,---,yp;dzi,...,dZp)^{zi,...,Zp) = J ^{dr) du^{r{l{u<yr}-yi),---,r{l{u<y^}-yp)). 

Consider now a family (z/("\ a > 0) of measures on ]0, 1] with /jg 1] r'^u^'^^dr) < 00, and for 
each a > 0, let F^") be the associated Fleming- Viot process. We then write 

f/-^) {x) := aF^^ {x/a) , xe[0,a],t>0 
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for the rescaled version of the Fleming- Viot flow. So, for each t > 0, F}"'' is the distribution 
function of a measure on ]0, a] with total mass a. For every fixed real number a > 0, we also 
denote by v^""^ the measure on ]0, oo[ which is on ]a, oo[ and whose restriction to ]0, a] is given 
by the image of u^""^ under the dilation r ^ ar from ]0, 1] to ]0, a]. In particular r^u^"'^ (dr) is a 
finite measure on ]0, oo[. 

By a scaling argument, we see that, for every {xi, . . . , Xp) G V^, {fI:"'\xi), . . . , Fi'^\xp)) is a 
purely discontinuous martingale, with values in D^, and the compensator of its jump measure 
is 

dz^... dzp) = dt i?(")(F/")(xi), . . . , F/")(xp); dzi, dzp) (11) 

where 

j R'^^'Kyi-, ■■■,yp; dzi, dzp) ip{zi, ...,Zp) 

= J u^"-\dr) ^ duip{r{l{u<y,} - a~^yi), . . .,r{l{u<y^} - a~^yp)). (12) 

Let TT be as in Section 2 a nontrivial measure on ]0, oo[ such that /(r A r^)7r(dr) < oo, and 
let ^ be as in (1). Denote by {Z{t,x),t > 0,x > 0) the associated flow of continuous-state 
branching processes constructed in Section 2. 

Assumption (H) The measures (r A r^)i'^"'\dr) converge to (r A r^)n{dr) as a — > oo, in the 
sense of weak convergence in 

Theorem 1 Under Assumption (H), for every {xi, . . . , Xp) G Vp, 

{{Ft\x,l f/")(x,)); t > 0) ^ ((Z(t, Xi), . . . , Z(t, xp))-t > 0) 
in the Skorokhod space D(IR.+ ,M^). 

Proof: The proof only uses the facts that (F/"^(a;i), . . . , Fi"-\xp)) is a purely discontinuous 
martingale and that the compensator of its jump measure is given by (11) and (12). The latter 
properties indeed characterize the law of the process {fI:"'\xi), . . . ,Fl:"'\xp)) (cf Lemma 1 in 
[5]), but we do not use this uniqueness property in the proof. We fix a sequence (a„) tending 
to -|-oo, and (a;i, . . . , Xp) G Vp. To simplify notation we write 

y," = {Yr'\ . . . , rn = (F^H^i), ■ ■ ■ , F,^^"\xp)) 

which makes sense as soon as a„ > Xp, hence for all n sufficiently large. We also set 

Zt = {Zl ...,Zf)^ {Z{t, x^),..., Z{t, Xp)). 

We rely on general limit theorems for semimartingales with jumps which can be found in 
the book [12]. To this end, we first need to introduce a truncation function /i : M — > R, that is 
a bounded continuous function such that h{x) = x for every x G [—5,5], for some 5 > 0. We 
may and will assume that h is nondecr easing, \h{x)\ < A 1 for every x G M and that h is 
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Lipschitz continuous with Lipschitz constant 1, that is \h{x) — h{y)\ < \x — y\ for every x, |/ e R. 
We can then consider the associated (modified) triplet of characteristics of the p-dimensional 
semimartingale y": 

(5",C",/i(<j^)). 

Sec Definition II. 2. 16 in [12]. To be specific, /i(a„) is defined in (11). Then, since 1^"' is a purely 
discontinuous martingale, we have = (i?"'*)i<j<p, with 

Bt'' = - / IJ'{an){dt, dzi... dzp) {zi - h{zi)) 

Similarly, = {Cl''''')i<i,3<p. with 

/ ix(^a^){dt,dzi. . .dzp)h{zi)h{zj). 

Write C*(]R^) for the space of all bounded Lipschitz continuous functions on W that vanish 
on a neighborhood of 0. We fix e C*(]R^) such that \g\ < 1, and we choose a > such that 
g{zi, . . . , Zp) = if l^il < a for every i — 1, . . . ,p. Following the notation in [12], we set 

(9 * l^{an))t = / l^{an){dt, dzi... dZp) g{zi, Zp). 

i[0,t]xKP 

Prom formula (11) we have 

Sr = f ds f5'''^\Y:^\ . . . ,Y^'^) 
Jo 

c^'''' = /'*ds7"'^'^(y,"'\...,y"'n (13) 

Jo 

{9*pi(ajt = l^ds^-{Yp^\...,Yr): 
where the functions /3"-'\ 7"^'''-?, cp'^ are defined by 

...,yp) = - / R^''"\yi, ...,yp; dzi, dzp) {zi - h{zi)) 

JMp 

7"«(yi,...,yp) = / R^''-\yi,...,yp-dzi,...,dzp)h{zi)h{zj) 

JWp 

¥'"(yi,---,yp) = / R^''"\yi,...,yp;dzi,...,dzp)g{zi,...,Zp). 

jRp 

Similarly, the (modified) characteristics of the semimartingale Z are 

{B,c,e) 

where ^ is as in Proposition 1, and 

Bi = f dsP\Zl...,Zl) 
Jo 

Cr = fdsY'^(Zl,...,Z^) (14) 
Jo 

{g*9)t = fdsip{Zl,...,Z^), 
Jo 
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where the functions P\ 7*'-^ , ip are respectively defined by 

P'iVi, ■■■,yp) = - / U{yi,...,yp;dzi,...,dzp){zi-h{zi)) 
f'^{yi,---,yp) = / U{yi,...,yp;dzi,...,dzp)h{zi)h{zj) 



fiVi, ■■■,yp) = / U{yi,...,yp;dzi,...,dzp)g{zi,...,Zp). 
Lemma 1 For every {yi, . . . ,yp) e Dp, 

< 2yijiy^a^){dr)rl{r>5} 
|7"«(|/i, ...,yp)\ < {yi + yj) j i/(a„)((ir) (r A r^) 



2 /■ 

< -yp n^ia„){dr)rl{r>a}- 



Moreover, 



Jim = p'{yi,...,yp) 

= 'p{yu---,yp), 

uniformly on bounded subsets ofVp. 

Let us postpone the proof of the lemma and complete that of the theorem. The first step 
is to check the sequence of the laws of the processes is tight in the space of probability 
measures on D(]R+,]R^). This will follow from Theorem VL4.18 in [12] provided we can check 
that: 

(i) We have for every N > and £ > 0, 

lim (limsupP[/X(„„)([0, AT] x e : \z\ > b}) > e]) = 0. 

(ii) The laws of the processes B"''\ C"''^'^ , g * ii(^a„) a-re tight in the space of probability measures 



on >_y - 

To prove (i), set 

T2 = inf > : F/'"' > A} 

for every A > Xp. Since Y'^'P is a (bounded) nonnegative martingale, a classical result states 
that 

P[sup{y;"'P, t>0}>A]= F[T2 < 00] < ^. (15) 
From formulas (11) and (12), we have on the event {sup{Yl'^,t >0}<A} 

/^(an)([0,A^] x{zeW: \z\ > b}) < iv(yli/(„„)(]-,oo[) + a„i/(„„)(]^,oo 
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Under Assumption (H), we have 



lim a„i/(a„)(] 

1— »0O ^ I' / 



pA 



oo[) = 



and so, on the event {sup{F/'",i > 0} < ^4}, 



h 

hmsup/i(„„)([0, A^] X e : \z\ > b}) < NAtt{[-,oo[). 




If we first choose A so that Xp/A is small, and then b large enough so that NA7r{[^, oo[) < £, 
we see that the statement in (i) follows from (15). Part (ii) is a straightforward consequence 
of formulas (13), the bounds of the first part of Lemma 1 and (15) again. This completes the 
proof of the tightness of the sequence of the laws of the processes F". 

Then, we can assume that, at least along a suitable subsequence, converges in distribution 
towards a limiting process Y°° = {Y°°'^, . . . ,y°°'^). We claim that is a semimartingale 
whose triplet of (modified) characteristics {B°° , C°° , jioo) is such that 



with 7*'-', if as above. To sec this, it is enough to verify that the 4-tuplcs ("K", B", C", g*^{ar,)) 
converge in distribution to (Y°° , , C°° , g * ^^o) (see Theorem IX. 2. 4 in [12]). The latter 
convergence readily follows from the convergence of towards Y°°, formulas (13) and the 
second part of Lemma 1. 

Finally, knowing the triplet of characteristics of Y°°, Theorem n.2.34 in [12] shows that Y°° 
is a purely discontinuous martingale, and the compensator of its jump measure is 



By Proposition 1, this implies that Y°° has the same distribution as Z, and this completes the 




(16) 



dt u{Yr\...,Yr^-dz,,...,dzp). 



proof of Theorem 1. 

Proof of Lemma 1: By definition, for (yi, . . . , yp) e D^", 



□ 



J jyM(dr) ^ du(r{l{^<y.y - a^^yi) - h{r{l{u<yi} - a^\))) 
J i/(»")(dr) ?/,(r(l - a-\) - h{r{l - a'^))) 




RecaUing that h{x) 



X if \x\ < 5, we immediately get the bound 
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Furthermore, using the fact that h is Lipschitz with Lipschitz constant 1, we have 

\(3'^'\y,,...,y,)+y, J v^^-\dT) {r - h{T))\ 

and it is easy to verify from Assumption (H) that the right-hand side tends to as n — cxd, 
uniformly when 7/j varies over a bounded subset in M_|_. Since Assumption (H) also implies that 

Jim J i/("")(dr) (r - h(r)) = J 7r{dr) (r - h(r)), 

we get the first limit of the lemma. 

Consider now, for {yi, . . . ,yp) G V^"-, and I < i < j < p, 

= j y^'^-\dr)y,h{r{l-a-'y,))h{T{l-a-'y^)) 

+ j v^^-\dT) {yj - y,) h{-a-'ryi) h{r{l - a'^yj)) 
+ j p^"-"\dr) {an-yj)h{-a~^ryi)h{-a~^ryj). 
Using the bounds \h\ < 1 and \h{x)\ < \x\, we get 

|7"«(l/i,...,l/p)| / u^''-\dr){r^Al) + y, j (c^r) r(r A 1), 
which gives the second bound of the lemma. Then, using the Lipschitz property of h, 
J u^^-\dr) h{r{l - a-^y,))h{r{l - a-^y^)) - J iy^^-\dr) h{rf 
< 2a-^yj J i^^''"'>{dr)rh{r) — > 
as n — > oo. Notice that 

lim yi [ z/("")((ir) h{rf = yi [ TT{dr) h{rf = Y'^Vu ■ ■ -^Vp): 



(17) 



uniformly when {yi,...,yp) varies over a bounded set. To complete the verification of the 
second limit in the lemma, we need to check that the last two terms in the right-hand side of 
(17) tend to as n — > oo. We have first 

I i/("")(dr) h(-a-^ryi)h(r(l - a'^)) < j v^''-\dr) ra'^y^ h(r) 

as n — >■ cxD. It remains to bound 

an J (c?r) h{-a-^ryi) h{-a-\yj) 

< I u^''-\dr)ryi{{a-^ryj)Al) 

<yi [ iy^''"\dr)rl{r>A} + yiyja-^ I iy^^"\dr) r\r<A} 
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where A > is arbitrary. If 77 > is given, we can first cfioose A sufficiently large so that 

limsup / i/^""-'(dr) rl{r>A} < / '^{dr) rl^r>A} < V- 

n— »oo J J 

On the other hand, we have also 

and together with the preceding estimates, this gives the second limit of the lemma. 
Finally, we have 

'P''{yi,---,yp) = J i^^^^Kdr) du g{r{l{u<y^y - a-^yi), . . . ,r{l{u<y^y - a-\)). 
Since Isfl < 1 and g{zi, . . . , Zp) — ii sup \zi\ < a, we easily get the bound 

< ypj ly^^-Kdr) l{r>a} + ^/ iy^''"\dr)rl{r>a} 
which gives the third bound of the lemma. Then, if M denotes a Lipschitz constant for g, 

V"{yi, ■■■,yp)- J z^^""^ {dr) dug{rl{u<y,}, rl{u<yp}) 
<MpJ u^^^^dr) ^ du{l^^<y^}a-^ryp 1{,>„| + l{u>y,}a-^ryp l{a-'ry^>a}) 
< Mp( j u^''-\dr)rl{r>a})a-Vp + MpypJ i^^'^'^Kdr) rl^^-i^^^^^y 

which tends to as n tends to 00, uniformly when yp varies over a compact subset of R+. The 
last convergence of the lemma now follows from Assumption (H). This completes the proof. □ 

4 Hydro dynamic limits for exchangeable coalescents 

The motivation for this section stems from hydrodynamic limit theorems leading from stochastic 
coalescents to Smoluchowski's coagulation equation, which we now summarize. 

4.1 Stochastic coalescents and Smoluchowski's coagulation equation 

Consider a symmetric measurable function K :]0,cx)[x]0, 00 [ — > M_|_ which will be referred to 
as a coagulation kernel. A stochastic coalescent with coagulation kernel K can be viewed as 
a Markov chain in continuous time C = {Ct, t > 0) with values in the space of finite integer- 
valued measures on ]0, oo[ with the following dynamics. Suppose that the process starts from 
some state Yli=i ^xi, where k > 2 and Xi e]0, oo[ for i = 1, . . . , A;. For 1 < i < j < k, let ejj 
be an exponential variable with parameter K{xi,Xj), such that to different pairs correspond 
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independent variables. The first jump of tlie process C occurs at time mini<j<j<fc ejj, and if 
this minimum is reached for the indices 1 < £ < m < k (i.e. i and k are the indices such that 
mini<j<j<fe ejj = Ge,m), then the state after the jump is 

In other words, a stochastic coalescent with coagulation kernel X is a finite particle system in 
]0, oo[ such that each pair of particles {xi, Xj) in the system merges at rate K{xi, Xj), indepen- 
dently of the other pairs. 

Now consider a sequence {Cl^\t > 0)„gN of stochastic coalescents with coagulation kernel 
K and set C[^^ = n^^C^jl for t > 0. Suppose that the sequence of initial states Cq"^ converges 
in probability in TWr to a Radon measure ^q. Then under some technical assumptions on the 
coagulation kernel K (see e.g. Norris [15]), the sequence {Ct'^\t > 0) converges in probability 
on the space of cadlag trajectories with values in A4r towards a deterministic limit {iit, t > 0). 
Moreover this limit is characterized as the solution to Smoluchowski's coagulation equation 

= I l^^p (/(^ + y) - m - m) K{x, y)Mdx)Mdy) , (18) 
where / :]0, oo[^ M denotes a generic continuous function with compact support. 



4.2 Hydrodynamic limits 

Let f denote a cr-finite measure on ]0, 1] such that /jg ij r'^h'{dr) < oo, and write A{dr) = r'^u{dr), 
which is thus a finite measure on ]0, 1]. The so-called A-coalescent (or coalescent with multiple 
collisions, see [16]) is a Markov process (nt,t > 0) taking values in the set of all partitions of 
N. Unless otherwise specified, we assume that Ho is the partition of N into singletons. For 
every t > 0, write Dt for the sequence of asymptotic frequencies of the blocks of 11^, ranked 
in nonincreasing order (if the number k of blocks is finite, then the terms of index greater 
than k in the sequence are all equal to 0). Then ([16], section 2.2) the process {Dt,t > 0) is 
a time-homogcncous Markov process with values in the space Si of nonincreasing numerical 
sequences s = (si, . . .) with J2ili ^ 1- 

The following connection with generalized Fleming- Viot processes can be found in [4]. Let 
{Ft,t > 0) be the generalized Fleming- Viot process associated with u, and for every t > 0, let 
Jt be the sequence of sizes of jumps of the mapping x Ft{x), ranked again in nonincreasing 
order, and with the same convention if there are finitely many jumps. Then, for each fixed 
t > 0, Jt and Dt have the same distribution (Theorem 1 in [4] indeed gives a deeper connection, 
which has been briefly described in Section 1). 

For each a > 0, let z>^"\ z/*^"\ F*^"^ and F*^") be as in Section 3. Denote by the point 
measure whose atoms are given by the jump sizes of the increasing process x f}"'\x): 



Fr{x)-Fr{x-y 



{xe]0,l]:F^''\x)-F^''\x-)>0} 
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Fom the preceding observations, the atoms of jxf' also correspond to the sizes of the blocks in 
a A-coalescent at time for A(dr) = r'^i'["^ {dr). We then consider the rescaled version //^"\ 
given as the image of a~^jl^at under the dilation r — > ar. Equivalently, jjS""^ is times the 
sum of the Dirac point masses at the jump sizes of the mapping x F^°'\x). 

Theorem 2 Suppose that (H) holds and let {Z{t,x);t,x > 0) be the flow of continuous- state 
branching processes associated with tt. Then for every t > 0, ixf^ converges to the Levy measure 
Xt of the subordinator Z{t, •) as a —> oo m probability in AIr. 

Theorem 2 is an immediate consequence of Theorem 1 and the following lemma. 

Lemma 2 Let a = {(Tt-,t > 0) be a subordinator with Levy measure A. For each a > 0, let 
X(a) = (xi"\0 <t <a) be an increasing cddldg process with exchangeable increments, with 
X^"-^ = and X^") = a a.s. Suppose that X^"'^ converges to a as a ^ oo in the sense of 
finite- dimensional distributions. Then the random point measure 

0<t<a 

converges to A in probability in A4r as a —>■ oo. 



Proof: Pick some nonnegative continuous function / :]0, oo[— > M with compact support and 
write 

c := / f{x)X{dx) . 

J]0.oc\ 



By the Lcvy-Ito decomposition for subordinators, the random point measure Y^Aat>o^{tA(^t) 
on M+x]0, oo[ is Poisson with intensity dt®X{dx). Let p > 0. The law of large numbers ensures 



the existence of a real number Op > such that 



E 



< E /(A<7,)-cl <p. (19) 



p 

0<t<ap 



Then consider for a > Op the bridges with exchangeable increments, bounded variation and 
no negative jumps on the time interval [0, Up] , defined by 

5{a).^^(a)_^^^-iX(«) , B, = at-ta;'aa^, te[0,ap]. 

Our assumptions entail that B^"-^ converges in the sense of finite dimensional distributions to 
B, so according to Kallenberg [13], the random measure 



0<t<ap 0<t<ap 



converges in law on AIr towards 



<t<ap 0<t<ap 
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and in particular, when a — > oo, 



a;^ E ^ < E . (20) 

0<t<ap 0<t<ap 

Let us check that the variables 

0<t<ap 

are uniformly integrable. Let [u,v] be a compact subinterval of ]0, oo[ such that the support 
of / is contained in [u,v]. Denote by N^u,v] ^he number of jumps of the process with size 
in [u,v]. By classical results about processes with exchangeable increments, conditionally on 
^[a ~ number 

0<t<ap 

has a binomial B{n, ^) distribution. Notice that A^[^ < ^ since X^") = a. We see that A^^^"'' 
is bounded above in distribution by a binomial ^) distribution, and the desired uniform 

integrability readily follows. 

It then follows from (19) and (20) that 



hm E 

a— »oo 



0<t<ap 0<t<ap 

Moreover, an easy exchangeability argument shows that we have also 



< 



limsupE[a-^ ^ f{Axi^^)-c 

0<t<a 



< 



Since p may be taken arbitrarily small, we have thus shown that 



t<a 



in for every continuous function / with compact support. The conclusion now follows by a 
standard argument. □ 

We will now show that the family {Xt,t > 0) of Levy measures, which appears in Theorem 
2, solves a certain coagulation equation with multiple collisions. To this end, we introduce the 
following additional assumption, which also plays a key role in the study of the genealogical 
structure of continuous- state branching processes (see e.g. [9]). 

Assumption (E) The ^-CSBP becomes extinct almost surely. 

Equivalently, this assumption holds iff P[Z(t,a:) = 0] > for every t > and x > 0. By 
solving (3), it is easy to verify that Assumption (E) is equivalent to 

Ji *(w) ^ ' 
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In particular, Assumption (E) holds in the so-called stable case \E'(m) = ^ 7 e]l, 2[, that will 
be considered in Section 5 below. 

Prom (4), we see that under Assumption (E) we have dt — 0, and the total mass At(]0, oo[) = 
— logP[Z(t, 1) = 0] is finite for every i > 0. Moreover the function t — > At(]0, oo[) is nonin- 
creasing. 

We denote by C,(M+) the space of all bounded continuous functions / on such that 
/(O) = and f{x) has a limit as a; ^ +00. The space C,(]R+) is equipped with the uniform 
norm, which is denoted by ||/||. For every integer k > 2 and q > 0, we denote by "^^^^q) the 
k-th derivative of at q'. It is immediately checked that 

= (-1)*^ J n{dr) rV«^ (22) 
Obviously, (-l)^\E'(^)(g) > for every k >2 and q> 0. 

Proposition 3 Under Assumption (E), for every f e C,(R+); the function t — > (At, /) solves 
the equation 



dt k=2 



^ A;! ^]o,oo['= 

(23) 



where the series in the right-hand side converges absolutely. 



It is interesting to observe that (23) also holds when ^{q) = cq for some constant c > 0. 
Take ^{q) = for definiteness (then the ^-CSBP is the classical Feller diffusion) in such a 
way that (23) exactly reduces to (18) with K =1. Then Ut{q) — 2q{2-\- qt)~^, and it follows 
that 

4 2x 

Xt{dx) = - exp( — -) dx (24) 
t 

so that the density of At is the classical solution, arising from infinitesimally small initial clusters, 
of the Smoluchovski equation (18) in the case K =1 [cl Section 2.2 of [1]). 

We can rewrite equation (23) in a somewhat more synthetic way by introducing the following 
notation. If is a measure on ]0, oo[ such that i]ooo[(l ^ x)fi{dx) < 00, we write yU® for the 
distribution on [0, oo[ of the sum of the atoms of a Poisson random measure on ]0, oo[ with 
intensity /x. Note that /x® is a probability measure and that, by Campbell's formula, 

/ e-«>®(dx) = exp I - / (1 - e-«^)/x(dx)| , q>0. (25) 

J[0,oo[ J]0,oo[ 

As we will see in the proof below, (23) follows from the equation 

d{Xt,f) 



dt 



[ n{da){{{aXtr,f)-{aXt,f)). (26) 

>/]0,oo[ ^ ^ 



Informally, we may think of Xt{dx) as the density at time t of particles with size x in some 
infinite system of particles. The right-hand side in (26) can be interpreted by saying at rate 
7r{da), a 'quantity' a of particles coagulates at time t. More precisely, this 'quantity' is sampled 
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in a Poissonian way, viewing at aXt as an intensity measure for the sampling (so, loosely 
speaking, the particles involved into the coagulation are sampled uniformly at random amongst 
the particles present at time t). 

As the proof below will show, (26) still holds without Assumption (E) at least for functions 
/ of the type f{x) = 1 — exp(— gx), provided that df = for every t > (recall from Silverstein 
[19] that the latter holds whenever J]o^i[T7:{dr) = oo). In that case however, the measures Aj 
may be infinite, and then coagulations involve infinitely many components, so that one cannot 
write an equation of the form (23). 

Proof: We first prove (26). For q>0, let f\g) G C(M+) be defined by f{g){x) = 1 - e""^. By 
(25) and (4), 

{{aXtf, /(,)) = 1 - exp ( - a I Xt{dr){l - e"'^^)) = 1 - exp(-aiit(g)). 
On the other hand, by (4) again, 

{Xt,f{q)) = ut{q). 

Thus when / — f(^q^ the right-hand side of (26) makes sense and is equal to 

/ Ti{da)(l - exp{-aut{q)) - aut{q)) = -'^{ut{q)). 

i]0,oo[ ^ ^ 

Therefore (26) reduces to (3) in that case. Note that we have not used Assumption (E) at this 
stage (except for the fact that dt — for every t > 0). 

Denote by H the subspace of C,(]R+) that consists of hnear combinations of the functions 

/(q). Then Ti is dense in C,(R+). Obviously, for every / G ?i, (26) holds, and the right-hand 
side of (26) is a continuous function of t g]0, oo[. Fix / G C,(]R+) and a sequence (/n)n>i in 
that converges to /. If we also fix < e < t, we have for every n > 1, 

(XtJn) = {K,fn) + jysJ n{da)[{{aXsf,fn) - (aKJn)). (27) 
Plainly, for every s > 0, 

and ((aA,)®,^)^-^((aA,)®,/). 

We claim that there exists a constant such that, for every s > e and n > 1, and every 
h G C.(M+), 

\{{aXsf,h) - {aXs,h)\ < ^(a^ Aa) \\h\\. (28) 

As the quantities (A^, 1), s G [e, oo[ are bounded above, it is clearly enough to consider a < 1. 
Since h{0) = 0, the definition of (aA^)® immediately gives 

((aA,)®, h) = e-«<^-i) a(A„ h) + 0{a'\\h\\) 

where the remainder 0(a^||/i||), which corresponds to the event that a Poisson measure with 
intensity aXg has at least two atoms, is uniform in h E C,(R+) and s > e. The estimate (28) 
follows. 
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Using (28) and dominated convergence, we get 

Jim I n{da) (((aA,)® , /„) - (aA„ /„)) = J n{da) (((aA,)®, /) - (aA„ /)) (29) 

uniformly in s e [e, oo[, and the right-hand side of (29) is a continuous function of s. Equation 
(26) in the general case follows by passing to the hmit n — > oo in (27). 

Then, to derive (26) from (23), we write 



= J 7r(rfa) (( £ ^ e-'^<^-i> J f{x, + ■ ■ ■ + x,) Xt{dx^) . . . Xt{dx,)) - a{Xt, /)) 

= J n{da) - e-«<^*'i) j (/(xi + • • • + x,) - {f{x,) + ■■■ + f{x,))) X,{dx,) . . . Xt{dxk). 

Notice that the term k — I'm the last series vanishes. Moreover, bounding the other terms by 
their absolute value gives a convergent series, whose sum is intcgrablc with respect to 7r{da). 
Hence we may interchange the sum and the integral with respect to 7r(cia), and we get the 
statement of the proposition from (26). □ 

Remark. To conclude this section, let us observe that Assumption (E) is closely related to the 
property for a A-coalescent to come down from infinity (cf Pitman [16] and Schweinsberg [18]). 
Let u denote a cr-finite measure on ]0, 1] such that /jq ^ r'^u{dr) < oo, and let A{dx) — x'^u{dx). 
Let be given by (1) with — v (and /3 = 0). Then the A-coalescent comes down from infinity 
if and only if the ^-CSBP becomes extinct almost surely. To see this, recall from Schweinsberg 
[18] that a necessary and sufficient condition for the A-coalescent to come down from infinity is 

j Al-rr'i^idr))- <oo. (30) 

Using the binomial formula, we can rewrite this condition as 

5^ ( / (6r - 1 + (1 - r)^) v{dr)) < oo, 

or equivalently, if we put $(g) = J{qr — 1 -|- (1 — r)^) z/(cir) for every real q > 1, 

dq 



$(g) 



< oo. (31) 



(note that the function $ is nondecreasing on [1, oo[). Simple estimates give the existence of a 
constant c e]0, 1[ such that, for every q >2, 

c*(g) < $(?) < *(g). 

It follows that (30) and (21) are equivalent. In the spirit of the present work, it would be 
interesting to give a direct probabilistic proof of the equivalence between the property for a 
A-coalescent to come down from infinity and Assumption (E) for the associated branching 
process. 
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5 Small time behavior of flows and coalescents 



In this section, we fix a measure v on ]0, 1] such that / r'^v{dr) < oo and we consider the 
associated generahzed Fleming- Viot process {Ft,t > 0). 

Prom now on until the end of the section, we make the following assumption on u. 

Assumption (A). The function iy{[s, 1]) is regularly varying with index —7 as £ — > 0, for some 

7e]l,2[. 

As a consequence, there exists a function L{s), s e]0, 1] that is slowly varying as £ ^ 0, such 
that, for every e e]0, 1], 

u{[e, 1]) = £-^L(£). 

Fix £0 > such that i^([£o, 1]) > 0. For e e]0, £0] we have L{e) > and so we can set 

e 

for < X < £~^ and t >0. We also let be the measure on [0, £'~^] defined by 

J u,{dr) ifir) = L(£)-^£^y' u{dr) ifC-). 

A simple scaling transformation shows that for every (xi, . . . , Xp) E T>p^^, (F^{xi), . . . , Ff{xp)) 
is a purely discontinuous martingale, with values in 1^^*^, and the compensator of its jump 
measure is 

dt R^{F^%xi), F^{xp)] dzi, dzp) 

where 



j Reivi, ■■■,yp; dzi, dzp) ip{zi, ...,Zp) 

= J J^s{dr) ' du <^(r(l{„<j^i} - £|/i), . . . , r(l{„<j^p} - eyp)). (32) 

Let TT-y be the measure on ]0, oo[ such that 7r^(]a, cxd[) = for every a > 0, and let 



We let {Z{t,x),t > 0,a: > 0) be the flow of continuous-state branching processes constructed 
in Section 2, with ^ = 

Theorem 3 Under Assumption (A), for every {xi, . . . , Xp) e T>p, 

m{x,), F^{xp))-t > 0) ^ {{Z{t, xi), . . . , Z{t, Xp))-t > 0) 



in the Skorokhod space 
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Proof: This is a simple consequence of Theorem 1, or rather of its proof. Indeed, we immedi- 
ately see that the kernel Rsivi, . . . ,yp; dzi, . . . , dzp) coincides with i?^^/^) {yi, . . . ,yp; dzi, . . . , dzp) 
defined in (12), provided we take From the observation at the beffinnine; of the 

proof of Theorem 1, we see that Theorem 3 will follow if we can check that Assumption (H) 
holds in the present setting, that is if 

lini(r A r^) Ueidr) = (r A r^) 7r^{dr) (33) 

in the sense of weak convergence in Aip. 

In order to prove (33), first note that when s —>■ 0+, 



X 



xMdx) = 2 ryu{[y,l])dy-eM[eA]) ~ ^e'-'Lie), 
Jo 2 — T 



[0,6] Jo 2-7 

where the equivalence follows from Assumption (A) and a classical property of integrals of 
regularly varying functions. We immediately deduce that 

limr'^ Us{dr) = r'^ 7i^{dr) (34) 

in the sense of vague convergence in the space of Radon measures on [0, oo[. 
Next, note that 

/roo roo 
v^{dr) {r — r /\ a) — / drv^{[r, oo\) — e^L{e)~^ j drv{[re,l\) — ^0 (35) 

uniformly in e e]0,£o]- From (34) and (35) the family ((r A r^)i'^{dr),0 < £ < 1) is tight for 
the weak topology in Mf- Together, with (34), this establishes the weak convergence (33). □ 

Remark. Suppose that {Ft,t > 0) is the flow of bridges associated with the Kingman coales- 
cent, corresponding to A = 5o in our notation (cf Section 4 in [5]). If we fix (|/i, . . . , yp) G V^, 
the process (Ft(yi), . . . , Ft{yp)) is a diffusion process in with generator 

i,j=l * j 

(see Theorem 3 in [5]). Putting F^{x) — -F^tisx), it is a simple matter to verify that our 
Theorem 3 still holds in that setting, provided we let {Z{t,x),t > 0,a; > 0) be the flow 
associated with the Feller diffusion (^(g) = |?^)- Indeed, if we specialize to the case p = 1 
and if we let {Bf, t > 0) be a standard linear Brownian motion, this is just saying that, for the 
Fisher- Wright diffusion {Xt{x),t > 0) solving 



dXt^^Xtil-Xt) dBt , Xo = x, 



the rescaled processes Xf := ^X^-t{ex) converge in distribution as £ — > towards the Feller 
diffusion Y^i^x) solving 

dYt^JVtdBt , Yo^x. 
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We will now use Theorem 3 to derive precise information on the sizes of blocks in a A- 
coalescent (for A{dr) — r'^u{dr)) in small time. As previously, we denote by Xi{dr) the Levy 
measure of the subordinator {Z{1, x),x > 0). We have for every q>0 

exp —X y (1 ~ e^''^)Xi{dr) = E(exp —qZ(l, x)) = exp —xui{q) 

and the function iti(g) can be calculated from equation (3), with ^ — ^j. It follows that 

jil- e-^'-)Ai(dr) = (r(2 - 7) + gi-T)V(i-7) 

and in particular, the total mass of Ai is 

(r(2-7))^/M. 

We will need the fact that Ai has no atoms. An easy way to derive this property is to argue 
by contradiction as follows. Suppose that a > is an atom of Ai. From the Levy-Khintchin 
decomposition of Z{t,x) (see the discussion after (4)), it follows that a is also an atom of the 
distribution of Z{l,x), for every x > 0. By a simple scaling argument, for every s > 0, the 
image of Xi{dr) under the mapping r — >■ s^/('>'~^)r is s^^^"'^^^Xs{dr). Therefore, for every s g]0, 1[, 
gi/(7-i)Q ig also an atom of Ag, hence of the distribution of Z{s,x) for every x > 0. However, 
applying the Markov property to the process {Z{t, l))t>o at time 1 — s, this would imply that 
for every s g]0, 1[, s^^^'^'^^a is an atom of the distribution of Z{1, 1), which is absurd. 

We set g{e) = L{e)~^e'^''^ for every e e]0, £o]- 

Theorem 4 Assume that (A) holds and let A{dr) = r'^u^dr). For every t > and r G [0, 00], 
denote by Nt{]0,r[) the number of blocks at time t with frequencies less than r in a A-coalescent 
started from the partition of N in singletons. Then, 



sup 

a;€]0,oo[ 



eNg^,){]0,ex[)-X,{]0,x[) 



— > 

£-*0 



in probability. 



Again Theorem 4 is a generalization of classical results for the Kingman coalescent. In that 
case, one has 



sup 

xG]0,oo[ 



£Ar,(]0,£x[) -2(l-2e-'^) 



— > 



almost surely (cf Section 4.2 of [1]). This is consistent with Theorem 4 since in the case 
^(9) = (24) shows that 



2(l-2e-'^)= r4e-2^dx = Ai(]0,x[). 
Jo 



Proof: By the results of [4] recalled at the beginning of subsection 4.2, we know that, for each 
t >0 the collection (A^t(]0, r[), r > 0) has the same distribution as 

( X] l{0<Ft(u)-Ft(«-)<r} , > 0), 
0<u<l 
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where {Ft, t > 0) is the generahzed Fleming- Viot process associated with u. It then follows 
from our definitions that 

[eNg^^){]0,xe[),x > o) = [e ^ l{o<Ff(«)-Ff(«-)<x} , a; > o). 

0<U<l/£ 

By combining Theorem 3 and Lemma 2, we get that 

e Yl ^ ^1 (36) 

0<U<l/£ 

in probabihty in Mr. This is indeed the same result as Theorem 2 in our present setting. The 
preceding convergence is not quite sufficient to conclude: Recalling that Ai has no atoms and 
using Dini's theorem, we see that the statement of the theorem will follow if we can prove that 
the convergence (36) holds in the sense of weak convergence in the space /Ap. To get this 
strengthening of (36), it suffices to prove the convergence of the total masses. Therefore the 
proof of Theorem 4 will be complete once we have established the following lemma. 

Lemma 3 We have 

lim£iV,(,)(]0,oo[) = Ai(]0,oo[) = {T{2 - 

in probability. 

Remark. The recent paper [2] gives closely related results that were obtained independently 
of the present work. 

Proof: Write Nt — A^t(]0, oo[) to simphfy notation. Then, for every t > and x e]0, 1], we 
have 

E[x^'] = F[Ft{x) = 1] 
(cf formula (8) in [5]). By exchangeability, 

¥[Ft{x) = 1] = ¥[Ft{x) = Ft{l)] = F[Ft{l - x) = 0]. 

Hence, for x e]0, 1[, 

F[Ft{x) =0] =E[{l-x)^% 

and it follows that 

F[F^{x) = 0] = E[(l - ea;)^«(-)]. 
Prom the convergence in distribution in Theorem 3, we have for every x > 0, 

limsupP[Ff(x) = 0] < F[Z{l,x) = 0] = exp -xAi(]0, oo[). 

£->-0 

We have thus obtained that, for every x > 0, 

limsupE[(l -£a;)^«w] < exp — a;Ai(]0, oo[). 

25 



By standard arguments, this imphes that for every r] > 0, 

limP[£Ar^(,) < Ai(]0, oo[) -ri]=0- (37) 

To complete the proof, we need to verify that we have also, for every i] > 0, 

limP[£Arg(,) > Ai(]0, oo[) + r;] = 0. (38) 

Fome now on, we fixe rj > and we prove (38). We will use a different method based on 

the knowledge of the law of the process of the number of blocks in a A-coalescent. For every 
integer n > 1, write N'l'' for the number of blocks at time t in a A-coalescent started initially 
with n blocks. Then according to Pitman [16] (Section 3.6), the process {N^,t > 0) is a time- 
homogeneous Markov chain with values in {1,2,..., n}, with only downward jumps, such that 
for 2 < A; < 6 < n, the rate of jumps from htoh — k-\-l\s 



The total rate of jumps from b is thus 

b 

o^h^Y. ^b^k = (1 - (1 - x)" - 6(1 - xf-^)u{dx). 

Lemma 4 Under Assumption (A), we have 

lim (6^L(l/6))-^Q;6 = r(2-7) 

6— »+oo 

and, for every integer k>2, 

hm {h'^L{l/h))-^a,,,^ ^^^\;^\ 
6-»+oo /e! 

We leave the easy proof to the reader. Note that 

g7r(^^^^2-7). (39) 

k=2 

This is easily proved by using the definition of the function F and then an integration by parts. 
Similarly, we have 

Let us fix p e]0, l/8[ sufficiently small so that 

(F(2 - 7)V(i-7) + ^)i-7 < (1 _ 6p)r(2 - 7). 

Thanks to (39) and (40), we may choose an integer K >2\/ Sq^ sufficiently large so that 

1 ^2£(^,,_, 



F(2 - 7) t^, k\ 
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and 

Then, for every k e {2,3, . . .,K}, we may choose pk e]0, 7r(A; — ^)/k\[ sufficiently small so 
that 

1 ^ 



r^,_^^Uk-l)p.<P. (42) 



Now set 



A,,^(2£(^-p,)rL(l) 



iorb>K and A; e {2, 3, ... , K}. We also put 

k=2 

Notice that 

/3p = E C^^\7 - Pkynl) > (1 - 2p)r(2 - 7) b-'Lil). (43) 

k=2 

By Lemma 4, wc can choose an integer B > 2K sufficiently large so that, for every b > B — K, 
b' e {b,b + 1, ... ,b + K} and k e {2, ... , K}, one has 

Pb',k < Oib,k- (44) 

Denote by {U^ )t>Q the continuous-time Markov chain with values in N, with initial value Uq — 
n, which is absorbed in the set {1, . . . , B — 1} and has jump rate (3b^k from btob — k + 1 when 
b > B and k e {2, 3, ... , K}. Fix n > B. Then thanks to inequality (44), we can couple the 
Markov chains {Up)t>o and {N^)t>o in such a way that 

C/f > A^f , for every i < Tg := inf{s : < B}. 

Now it is easy to describe the behavior of the Markov chain (t/"). Note that for k G 

{2, . . . ,K} and b > K the ratio Pb,k/Pb does not depend on b. Then denote hy Si = H h^i 

{i — 0,1,2, . . .) a discrete random walk on the nonnegative integers started from the origin and 
with jump distribution 



P[^. = k-i] = t^ = ' , 2<k<K. 



f3b,k (-r(/,- 

Pb Ef=2((7r(^-7)/^!)-p^) 

Prom (41) and (42) we have 



mi] > - 2p. (45) 

7-1 

Let Co, ei, . . . be a sequence of independent exponential variables with mean 1, which are also 
independent of the random walk {Si)i>Q. We can construct the Markov chain {U^) by setting: 

C/f = n if < i < 

TTn C -r ^0 , , ^ eo 61 

= n — Si if — < t < — + 



Pn Pn Pn-Si 
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and more generally, 



H h 



ep-i 



<t<-^ + 



eo , ei 



0n Pn-Si Pn-Sp-i Pn Pn-Si 

provided p <p% := inf{i : n — Si < B}. 

Recall that our goal is to prove (38). To this end, note that for a > B, 



H h 



'n—S„ 



P[7V" > a] < Fm,) > a] < P g{e) < + 



eo , ei 



/-'n yn—Si 

where := inf {i : n — Si < a}. 

Lemma 5 For e > set a{e) = (Ai(]0, oo[) + r])/e. Then, 

eo , ei 



H h 



Pn-S„n 



(46) 



sup P g{e) < — 

- - n>a{e) 



e«". 



H h 



a(e) 



)=0. 



The desired bound (38) immediately follows from Lemma 5. Indeed standard properties of 
A-coalescent give 

P[£7V^(,) > Ai(]0, oo[) +r)]^ hm T n^N^ie) > Ai(]0, oo[) + r)] ^ lim T P[A^;(,) > a{e)] 

and by combining (46) and Lemma 5, we see that the latter quantity tends to as £ — >■ 0. 
Proof of Lemma 5: By (43), we have for a > B, 



eo _^ ei 



Pa 



B ■ /5 +--- + ]r^^((i-2p)r(2-7))-^E ., owVr ^ V 



(47) 



Note that 



Pa 



Pa 



i(ri-Si)'^L{^y 



Let m > 2 be an integer. For a > B and n > ma, a trivial bound shows that 

Pma 1 -| CXD 



"'"^^^^ g (n - 5.)-L(^) 



1 1 ^ 

^<a-^L(i) E 



and the right-hand side tends to as m — > cxd, uniformly in a > 5. On the other hand, an easy 
argument using the law of large numbers for the sequence {Si)i>Q shows that, for each fixed 
m > 2, 

lim f sup E a?-^Li-) V , , - / — ) = 0. 

Now recall the bound (45) for E[^i]. It follows from the preceding considerations that 



lim fsupPfaT~^L(-)y 



> 



0. 



(48) 
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Now we can also get an estimate for the conditional variance 



var 



fa 



p 

^^,^>o) = x: 



1 



< E 



1 



3=[a-K] 



a 

for some constant C independent of a and n. Prom this estimate, (48) and an application of 
the Bienayme-Cebycev inequality, we get 



lim ( sup P 

o->-oo V n>a 

Recalling (47), we arrive at 
limf inf P 

£-^0 V n>a{e) 



1 Pa 

a--^L(i)^ 



> 



(49) 



eo ei 

— H = \ h 



f-aie) 



Pn- 



< 



(l-2p)(l-4p)r(2-7) 



However, from our choice of p, we have for e sufficiently small 



9{e) > 



(l-2p)(l-4p)r(2-7)' 



and this completes the proof. 



□ 



Remark. It is rather unfortunate that the simple argument we used to derive (37) does not 
apply to (38). On the other hand, it is interesting to observe that the techniques involved in 
our proof of (38) would become more complicated if we were trying to use them to get (37). 
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